The present invention relates to an image processing method and apparatus, more particularly, to an image processing apparatus for compensating a deterioration of an image quality of an image representing image data, which occurs due to lens characteristics, for the image data of the image obtained by reading a recording material on which the image is projected and recorded through a lens. More specifically, the present invention relates, in a digital photo printer, etc. for obtaining digital image data by photoelectrically reading an image photographed on a film or an image directly as digital image data by photographing with an image pickup device and a print (photograph) reproducing the thus obtained image as digital image data, to an image processing method and apparatus based on a technical field of compensating aberrations such as a transverse chromatic aberration and a distortion aberration due to a photographic lens, which occur in an image captured by a lens-attached film, an inexpensive compact camera, or a digital camera, etc.
Currently, printing on a photosensitive material (photographic printing paper) an image photographed on a photographic film such as a negative film and a reversal film (hereinafter referred to as a film), has been executed by a so-called direct exposure (analog exposure) of exposing a surface of the photosensitive material to the light by projecting the film image upon the photosensitive material.
By contrast, there has been hitherto also known an image processing system in which for image data obtained by reading a film image recorded on a photographic film or image data inputted from a digital camera, etc., after performing various image processes, it is capable of outputting the image in various output forms such as recording the image to a recording material, e.g., a photographic printing paper, displaying the image on display means, e.g., a display, and storing the image data into an information recording medium. According to this image processing system, as compared with the conventional photographic processing system for recording the film image to the photographic printing paper through the surface exposure, an image quality of output image from the image processing system can freely be controlled by the image processing for the image data, so that the output image having a high quality can be realized.
That is, in this image processing system, there has been in recent years utilized a printing apparatus that makes use of a digital exposure, i.e., a digital photo printer for obtaining a (finished) print by photoelectrically reading an image recorded on the film, converting the read image into digital signals, thereafter conducting a variety of image processing on the digital signals to thereby acquire output image data for recording, and recording the image (latent image) by scan-exposing the photosensitive material to recording light modulated in correspondence with the output image data.
The digital photo printer is capable of converting the image into digital image data and determining an exposure condition upon printing through the image data processing and is therefore able to obtain a high-definition print, which could not be acquired so far by the prior art direct exposure method, by preferably compensating a white compression and a black compression which might be attributed to back light and flash photography, etc., executing a sharpness (sharpening) process, compensating a failure in terms of shortage of a peripheral light quantity and the like.
In case that a distortion occurs in the image photographed and recorded on the film, however, the image quality of the output image outputted as a photo print cannot often be improved by performing the above-mentioned compensation. As the cause of the image distortion, there might be exemplified a transverse chromatic aberration and a distortion aberration arisen from a lens performance attached to the camera that has photographed the image.
A color image is formed by three primary colors of red (R), green (G), and blue (B). Since indexes of refraction (image forming magnifications) of the lens are fine different by a wavelength, the image forming magnifications of R, G, and B lights are various, namely, the transverse chromatic aberration occurs. As a result, if the image photographed on the film is reproduced, a color deviation is caused to the obtained image.
In order to acquire a preferable photographed image, it is necessary that with respect to a plane vertical to an optical axis, an image is formed on the image-forming surface corresponding to it. However, as for a normal lens, the image-forming position deviates in the optical axis direction and a distortion, that is, a distortion aberration is caused in the image-formed image, so that when the image photographed on the film is reproduced, the obtained image is distorted.
In case of using a camera for which costs to some degree might be increased such as a single lens reflex camera, by using a lens having a high precision and further by combining a plurality of lenses, various aberrations such as a transverse chromatic aberration and a distortion aberration are compensated and thereby enabling a proper image to be photographed on the film. Nevertheless, a lens-attached film or a compact camera, etc. might be severely restricted in terms of cost for the lens, with a result that the transverse chromatic aberration and distortion aberration are caused in the image photographed on the film. The image reproduced as a print might accordingly have distortions.
To solve the problem of the image quality deterioration that the image quality of the output image outputted as a photo print cannot be improved, techniques in relation to an image processing method of and an image processing apparatus for compensating an aberration of the image according to lens aberration characteristics obtained through lens information obtaining means have been disclosed in Japanese Patent Application Laid-Open No. Hei 6-311425 and Japanese Patent Application Laid-Open No. Hei 9-281613 (Japanese Patent Application No. Hei 8-92804). It is pointed out that by those techniques, aberrations due to a lens can be compensated, a deterioration of image quality in periphery of the image can be prevented, and thus an image having a high quality always can be acquired.
As mentioned above, yet, the lens of the lens-attached film is generally constructed by an inexpensive plastic lens, so that the aberrations such as a distortion aberration and a transverse chromatic aberration are large and in the film image exposed and recorded on the photographic film by the lens-attached film, a geometric distortion due to the lens distortion aberration and a color blur due to the lens transverse chromatic aberration occur relatively remarkably. Incidentally, with regard to the lens of the lens-attached film, a so-called drop in light quantity (decrease in light) in the periphery conspicuously occurs and there is also a problem that in the film image exposed and recorded on the photographic film by the lens-attached film, lightness in periphery of the image is more largely decreased than that in the center portion of the image. In the foregoing image processing system, hence, to obtain an output image having a high quality from the above-mentioned image, there is underway an examination of a contrivance that a distortion aberration compensation for compensating the geometric distortion in the image due to the lens distortion aberration, transverse chromatic aberration compensation for compensating the color blur in the image due to the lens transverse chromatic aberration, or a peripheral light decrease compensation for compensating the decrease in lightness in the periphery of the image due to drop in light in the periphery of the lens is executed.
The distortion aberration compensation can be performed as follows. When, for instance, an inherent position (lattice point position) of each pixel constituting a film image is set as a reference, distortion aberration compensating data indicative of the moving direction and moving amount at the position of each pixel due to the lens distortion aberration is preliminarily measured and stored every lens kind. For the image data as a processing target, the distortion aberration compensating data corresponding to the lens kind used in photographing is captured. On the basis of the captured distortion aberration compensation data, each pixel position, which is indicated by the data of each pixel in the case where the distortion aberration is absent, is decided. The density value at the inherent position (lattice point position) is obtained by an interpolation calculation.
The transverse chromatic aberration compensation can be further performed as follows. For example, at each position on the film image, transverse chromatic aberration compensating data, which shows the color deviation direction and the color deviation quantity of non-reference colors (e.g., R or B) for reference color (e.g., G) due to the lens transverse chromatic aberration, is preliminarily measured and stored every lens kind. For the image data as a processing target, the transverse chromatic aberration compensating data corresponding to the lens kind used in photographing is captured. On the basis of the captured transverse chromatic aberration compensating data, the position of each pixel, which is indicated by each pixel data without the transverse chromatic aberration, is decided every non-reference color. The density value at the inherent position (position similar to the that of the pixel of the reference color) is obtained by the interpolation calculation.
Furthermore, the peripheral light decrease compensation can be performed as follows. For instance, peripheral light decrease compensating data, which expresses a decreasing light quantity at each position on the film image, is preliminarily measured and stored every lens kind. For the image data as a processing target, the peripheral light decrease compensating data corresponding to the lens kind used in photographing is captured. On the basis of the captured peripheral light decrease compensating data, a fluctuating quantity in density value due to the peripheral light decrease is decided every pixel. The density value in the case where there is no peripheral light decrease is calculated every pixel.
However, when information regarding a photographic lens is not preliminarily acquired or the aberration characteristics of the photographic lens is not acquired, the compensation of the distortion aberration and transverse chromatic aberration due to the photographic lens cannot be realized by the above-mentioned techniques and the deterioration of the image quality cannot be prevented. Even if the aberration characteristics of the lens is acquired and the image having a high quality and no distortion on the basis of a compensating function and a compensating expression, the compensation of the distortion aberration is performed, so that a rectangular image 2 as shown in FIG. 18 becomes an image 4a of a pincushion form or an image 4b of a barrel form are obtained, with the compensation for the aberration characteristics done. Since the rectangular image thus becomes the image 4a of the pincushion form or image 4b of the barrel form, in order to output a predetermined rectangular image as an output print, a region 6a or outer region 8a in the image 4a of the pincushion form, or a region 6b or external region 8b in the image 4b of the barrel form needs to be outputted. On the contrary, the outer region 8a and 8b are not preferable as an output print image, because a hatched portion having no image in the output-printed image, a so-called vignetting of the image occurs. Preferably, the region 6a or 6b is enlarged to a desired print size and then outputted as a print image.
Although the inherent image, which was photographed, lies within a range of the region 4a or 4b, the region 6a or region 6b is outputted as a photo print. Thus, though the photographing was executed, the region which is not outputted as a photo print, namely, the lack of the image, a so-called vignetting of the image exists.
Particularly, if image information which is important in the region, for example, a main subject to be photographed such as a person has been photographed, the image is not outputted as a photo print and there are many cases where the head is separated in the image. If the object has been photographed by an inexpensive compact camera, etc. with a large aberration of the lens, the compensation quantity of the distortion aberration is large, so that though the photographing was executed, the region, which is not outputted as a photo print, is large and there are also many cases where the image information is not fully outputted as a photo print.
As discussed above, with respect to the compensation of the image quality deterioration due to the lens characteristics such as the distortion aberration compensation, transverse chromatic aberration compensation, and peripheral light decreasing compensation, the compensation quantity is obtained every pixel and the compensation is performed using each pixel as a unit basis. Data of each pixel, of which image data consists, therefore, has to be accurately corresponded to the compensation quantity every pixel obtained from each compensating data. If the correspondence is inaccurately executed, the compensation is performed under the compensation quantity which is different from the compensation quantity predetermined for each pixel data. Thus, the geometric distortion of the image representing the image data, color blur, and drop in lightness in the periphery of the image are unsolved. It is additionally fear that the geometric distortion, color blur, and drop in lightness in the periphery of the image might be oppositely large. In fact, there is not established a technique, in terms that in order to correspond accurately and easily, for the image data obtained by reading the image, using by which reference, data of the individual pixel might be corresponded to the compensating amount of every pixel.
To solve the above-mentioned problems of the prior art, it is a primary object of the present invention to provide an image processing method and an image processing apparatus for executing the method wherein even if lens characteristics cannot be obtained, aberrations such as a transverse chromatic aberration and a distortion aberration can be compensated and further, a proper compensating process of the aberrations can be performed in consideration of a vignetting of an image so that important image information such as a main subject to be photographed is necessarily outputted as a photo print.
It is a secondary object of the present invention, which might be accomplished in view of the above-mentioned fact, to obtain an image processing apparatus, in which for image data obtained by reading an image recorded on a recording material, a deterioration of image quality due to the lens characteristics can be compensated with a high precision.
To attain the primary object, according to a first aspect of the present invention, there is provided an image processing method, characterized in that:
obtaining input image data from an image optically photographed by using a photographic lens; and
executing a compensating process using position information of the image for at least one aberration of a chromatic aberration of magnification and a distortion aberration of the input image data; wherein:
a compensating level of the aberration is first set to an initial value, an aberration compensation result is displayed on a monitor after subjecting the input image data to the compensating process for the aberration, and an compensating level is allowed to be capable of designating with respect to the initial value at a plurality of stages; and
each time when one of the plurality of stages is designated, the compensating process for the aberration is again executed for the input image data according to the compensating level of a designated stage, and the aberration compensation result is displayed on the monitor.
In this instance, it is preferable that the initial value is a default value, and the compensating level is designated in such a manner that a plurality of compensating coefficients of a compensating function for at least one aberration of the chromatic aberration of magnification and the distortion aberration, which is preset in accordance with the default value or an aberration pattern corresponding to the image, are prepared for the default value, and one of the plurality of compensating coefficients is designated.
It is also preferable that the initial value is a compensating value which is automatically determined by pixel position information of the image and lens characteristics of the photographic lens, or a compensating value which is determined by an instruction of the operator; and the compensating level is designated in such a manner that a plurality of compensating coefficients of a compensating function for at least one aberration in compensating functions of the chromatic aberration of magnification and the distortion aberration, which is set according to the lens characteristics of the photographic lens, or designated by an instruction of the operator, are prepared for the compensating value and one of the plurality of compensating coefficients is designated.
It is further preferable that the compensation of at least one aberration of the chromatic aberration of magnification and the distortion aberration is performed in at least one direction of a first direction of the image and a second direction orthogonal to the first direction. Furthermore, preferably, the aberration is subjected to the compensating process in accordance with the designated compensating level, the compensation result displayed on the monitor is canceled and returned to the image data in a state of the aberration compensation result in which the compensating level is the initial value or the input image data in a state before the aberration compensation.
It is still further preferable that an unprintable quantity, which is caused as a result of the aberration compensation, is calculated in accordance with the designated compensating level, to display on the monitor a printable effective area together with the aberration compensated image.
It is yet further preferable that, prior to the compensating process for at least one aberration of the chromatic aberration of magnification and the distortion aberration, the image before the aberration compensating process is displayed on the monitor on the basis of the input image data and a print area is designated;
the compensating process of the aberration, for which the compensating level is changed at a plurality of stages in the designated print area, is executed; and
the compensating process, in which the aberration compensation is maximum, is executed.
It is also still further preferable that the initial value is zero.
According to a second aspect of the present invention, there is provided an image processing method, comprising the steps of:
obtaining input image data from an image optically photographed by using a photographic lens; and
executing a compensating process using position information of the image for at least one aberration of a chromatic aberration of magnification and a distortion aberration of the input image data, wherein:
the compensation of the aberration is executed by designating a print area from the photographed image region, by calculating a compensating coefficient of a compensating function of the aberration from a vignetting quantity of image, which is determined from the designated print area, and by using the calculated compensating coefficient and the compensating function of the aberration.
According to a third aspect of the present invention, there is provided an image processing apparatus, in which input image data is obtained from an image optically photographed by using a photographic lens and a compensating.process is executed using position information of the image for at least one aberration of a chromatic aberration of magnification and a distortion aberration of the input image data, comprising:
an image display device for displaying the image after the aberration has been compensated on the basis of the input image data;
means for obtaining an initial value of a compensating level of the aberration;
means for designating the compensating level obtained by the obtaining means at a plurality of stages; and
means for compensating the aberration according to the initial value of the compensating level obtained by the obtaining means or the stage of the compensating level designated by the designating means,
characterized in that the compensating process for the aberration is performed by the compensating means in accordance with the initial value of the compensating level obtained by the obtaining means, to display the compensation result on the image display device, and thereafter,
the designating means designates the stage of the compensating level from the displayed compensated image, and performs the compensating process for the aberration according to the designated stage of the compensating level, to display the compensation result on the image display device.
However, in the compensation for deterioration of the image quality due to the lens characteristics, which is the secondary object, it is considered that as for a reference that data of a plural kind, which has a 2-dimensional stretch, can be easily corresponded, it is exemplified that if an outer edge of an image recorded to a recording material is rectangular, a corner portion or one side among four sides composing the outer edge of the image is used as the reference and data of the plural kind is accurately corresponded to the reference. In particular, in a simple camera (projecting and recording apparatus) such as a lens-attached film, there is a case where a shape of the outer edge of the image recorded to the recording material, e.g., a photographic film, etc. is distorted and even if the corner portion or side is used as a reference, data of the plural kind cannot be corresponded. The inventor of the application concerned, in consequence that an experiment has been implemented by using various references as a reference of the correspondence, has confirmed that if the center position of the image is used as a reference, the compensation (compensation based on a pixel unit basis) for the image quality deterioration due to the lens characteristics is performed for most images with high precision.
Under the foregoing, an image processing apparatus according to a first embodiment in a fourth aspect of the present invention, comprising:
calculating means for discriminating a picture position of an image on a recording material on the basis of data obtained by reading the recording material to which the image has been projected and recorded through a lens, and for calculating a center position of the image on the basis of the determined picture plane position;
means for obtaining information relating to characteristics of the lens; and
means for compensating, on the basis of the information relating to the characteristics of the lens obtained by the obtaining means, a deterioration of an image quality due to the characteristics of the lens in the image indicated by image data with the center position of the image calculated by the calculating means as a reference for the image data of the image.
According to the present aspect of the invention (the first embodiment), the calculating means decides the picture plane position in the image on the recording material on the basis of data obtained by reading the recording material, on which the image has been projected and recorded through the lens, and calculates the center position of the image on the basis of the determined picture plane position. The center position in the image can be acquired by a relatively simple calculation. For instance, as shown in a second embodiment of the present aspect of the invention, on the basis of the determined picture plane position, the calculating means calculates a position, as a center position in the image, the position along a first direction almost parallel to two sides among four sides constituting an outer edge of the picture plane being located at a center of both edge portions along the first direction in the picture plane, and a position along a second direction orthogonal to the first direction, which locates at a center of both edge portions along the second direction in the picture plane.
The obtaining means obtains the information concerned with the characteristics of the lens used when the image has been projected and recorded on the recording material. It is sufficient that the information in relation to the lens characteristics is information showing the lens characteristics themselves, information showing the compensating value for compensating the image data according to the lens characteristics, information showing a lens kind, or information showing a kind of apparatus (camera, etc.) whereby the image has been projected and recorded on the recording material. As information in relation to the lens characteristics, if information showing the lens kind, or information showing the kind of projecting and recording apparatus (the lens kind can be decided from the information) is obtained, the lens characteristics and the compensating value for compensating the image data according to the lens characteristics can be decided indirectly from the obtained information.
On the basis of the information i n relation to the lens characteristics obtained by the obtaining means, the compensating means compensates the deterioration of the image quality due to the lens characteristics in the image, which the image data indicates, by using, as a reference, the center position in the image calculated by the calculating means, for the image data of the image [which can be data extracted out from the data obtained by reading the recording material (data used in the decision of the picture plane position and the calculation of the center position) or can be data obtained by again reading the image]. The compensation, which uses the center position of the image as a reference, can be executed as follows, specifically. For example, coordinates of each pixel of the image data is standardized by use of the center position in the image as a reference. For the image data after standardization, the compensating quantity for each pixel is decided on the basis of the coordinates of each pixel. The compensation can be performed on a pixel unit basis.
As described above, in the present aspect of the invention, the center position of the image is calculated and the deterioration of the image quality due to the lens characteristics, by using the calculated center position as a reference, is compensated. Hence, the deterioration of the image quality due to the lens characteristics can be compensated with a high precision for the image data obtained by reading the image recorded on the recording material.
In a third embodiment of the present aspect of the invention, it is preferable that the calculating means, on the basis of preliminary read data obtained by preliminarily reading the recording material by a predetermined resolution, calculates the center position of the image, and
the compensating means performs the compensation for main reading data, obtained by a main reading operation for reading the recording material by a resolution higher than the predetermined resolution.
When image contents of an image which has been projected and recorded on a recording material are indefinite or the like, to precisely read a recording material (image), there are many cases where the recording material is preliminarily read by a relatively low resolution, a reading condition upon main reading operation is determined on the basis of the preliminary read data obtained by the preliminary reading operation, and the main reading operation for reading the recording material by a relatively high resolution according to the determined reading condition is performed. As mentioned above, when the preliminary reading operation and main reading operation are executed for the recording material, as shown in the third embodiment, if the center position of the image is calculated on the basis of the preliminary read data , the center position in the image can be calculated for a short period of time.
Although with regard to the decision of the picture plane position, various methods have been conventionally devised, if any one method is used, the picture plane position often cannot be decided accurately (e.g., an image which has been exposed and recorded to a negative film through an over-under exposure). Therefore, according to a fourth embodiment of the present aspect of the invention, in the first embodiment, there is provided an image processing apparatus further comprising:
display means for displaying the image;
display control means for displaying the decision result of the picture plane position obtained by the calculating means on the display means; and
input means for inputting information instructing a compensation for the decision result of the picture plane position displayed on the display means,
characterized in that when the information instructing the compensation for the decision result of the picture plane position is inputted through the input means, the calculating means compensates the decision result of the picture plane position according to the instruction and calculates the center position on the basis of the compensated picture plane position.
In the fourth embodiment, the decision result of the picture plane position by the calculating means is displayed to the display means and if the information instructing the compensation for the decision result of the picture plane position is inputted through the input means, the decision result of the picture plane position is compensated according to the instruction and the center position is calculated on the basis of the compensated picture plane position. Consequently, even if the picture plane position of the image is erroneously decided, decrease in compensating precision for the deterioration of the image quality due to the lens characteristics can be prevented.